An Information-Theoretic Study on Variable-Length Source Coding with Unequal Cost

– Abstract – In the variable-length source coding, we may consider vary zestful problem as follows: If each code symbol is given prescribed cost to transmit or record, what code is optimal? This problem is substantial from not only theoretical viewpoints but also engineering points of view. Because, in practical systems of communication or recording , it may reasonable to consider unequal costs on code symbols. The source coding in the case where each code symbol has cost is called the source coding with cost. The problem of the source coding with cost was formulated by Karp, and has been studied by a lot of researchers to this day. The main aim of this thesis is to experiment relative questions to this problem. In Chapter 3, we shall inquire the relation between source codes with cost and nonuniform random number generations. First, we shall introduce the problem of random number generation and survey the history of it. Next, we shall give Han's previous result which states that optimal variable-length codes can be considered as uniform random number generators. Then, we shall manifest that optimal variable-length codes with cost can be regarded as nonuniform random number generators. In Chapter 4, we shall study the large deviation performance of the overflow and the underflow probabilities of codeword cost as well as codeword length. Firstly, we shall summarize the previous results of the large deviation performance with fixed-length coding for general sources established by Han. Secondly, we shall demonstrate that the infimum of achievable coding rate thresholds given the overflow probability exponent r (resp. the infimum of achievable coding rate thresholds given the underflow probability exponent r) always coincides with the infimum of achievable fixed-length coding rate given the error probability exponent r (resp. the infimum of achievable fixed-length coding rate given the correct probability exponent r). Moreover, we shall generalize above results to the case with the overflow and the underflow probabilities with cost. Finally, we shall state several examples of computations for both the infimum of achievable coding rate thresholds given an overflow probability exponent and the infimum of achievable coding rate thresholds given an underflow probability exponent. In Chapter 5, we shall propose two universal lossy source codes with cost: one minimizes the cost of the codeword when the maximum distortion is specified, and the other minimizes the distortion when the cost per source symbol is specified. In Section 5.1, …